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Abstract 

A remarkable similarity in the behavior of the US S&P500 index from 1996 to August 
2002 and of the Japanese Nikkei index from 1985 to 1992 (11 years shift) is presented, with 
particular emphasis on the structure of the bearish phases. Extending a previous analysis of 
Johansen and Sornette [1999, 2000] on the Nikkei index "anti-bubble" based on a theory of 
cooperative herding and imitation working both in bullish as well as in bearish regimes, we 
demonstrate the existence of a clear signature of herding in the decay of the S&P500 index since 
August 2000 with high statistical significance, in the form of strong log-periodic components. 
We offer a detailed analysis of what could be the future evolution of the S&P500 index over the 
next two years, according to three versions of the theory: we expect an overall continuation of 
the bearish phase, punctuated by local rallies; we predict an overall increasing market until the 
end of the year 2002 or at the beginning of 2003 (first quarter); we predict a strong following 
descent (with maybe one or two severe up and downs in the middle) which stops during the first 
semester of 2004. After this strong minimum, the market is expected to recover. Beyond, our 
prediction horizon is made fuzzy by the possible effect of additional nonlinear collective effects 
and of a real departure from the anti-bubble regime. The similarities between the two stock 
market indices may reflect deeper similarities between the fundamentals of two economies 
which both went through over-valuation with strong speculative phases preceding the transition 
to bearish phases characterized by a surprising number of bad surprises (bad loans for Japan 
and accounting frauds for the US) sapping investors' confidence. 

1 Introduction 

Financial crashes have been proposed to be critical phenomena in the statistical physics sense of 
critical phase transitions (see [23, 10, 16, 9, 22, 20] and references therein). Two hallmarks of crit- 
icality have been documented: (i) super-exponential power law acceleration of the price towards a 
"critical" time t c corresponding to the end of the speculative bubble and (ii) log-periodic modula- 
tions accelerating according to a geometric series signaling a discrete hierarchy of time scales. 

Imitation between investors and their herding behavior not only lead to speculative bubbles with 
accelerating overvaluations of financial markets possibly followed by crashes, but also to "anti- 
bubbles" with decelerating market devaluations following market peaks. There is thus a certain 
degree of symmetry between the speculative behavior of the "bull" and "bear" market regimes. 
This degree of symmetry, after the critical time t c , corresponds to the existence of "anti-bubbles," 
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characterized by a power law decrease of the price (or of the logarithm of the price) as a function 
of time t > t c , down from a maximum at t c (which is the beginning of the anti-bubble) and by 
decelerating/expanding log -periodic oscillations [11]. The classic example of such an anti-bubble 
is the long-term depression of the Japanese index, the Nikkei, that has decreased along a downward 
path marked by a succession of up and downs since its all-time high of 31 Dec. 1989. Another good 
example is found for the gold future prices after 1980, after its all-time high. The Russian market 
prior to and after its speculative peak in 1997 also constitutes a remarkable example where both 
bubble and anti-bubble structures appear simultaneously for the same t c . This is however a rather 
rare occurrence, probably because accelerating markets with log-periodicity often end-up in a crash, 
a market rupture that thus breaks down the symmetry (t c —t for t < t c into t — t c for t > t c ). Herding 
behavior can occur and progressively weaken from a maximum in "bearish" (decreasing) market 
phases, even if the preceding "bullish" phase ending at t c was not characterized by an imitation 
run-away. The symmetry is thus statistical or global in general and holds in the ensemble rather 
than for each single case individually. 

In [11, 12], the decrease of the Nikkei index has been analyzed in details, starting from 1 Jan. 
1990, using three increasingly complex formulas, corresponding to the three successive orders of a 
Landau expansion around of the logarithm of the price: d = aF(x) + ■••, where in general 

the coefficients may be complex. The first-order expression based on discrete scale invariance [19] 
of stock indices reads: 

lnp (t) taA 1 + B lT a + dr a cos [u In (r) + fa] , (1) 

where 

r = t-t c , (2) 

where t c is the time of the beginning of the anti-bubble. The inclusion of a non-linear quadratic 
term in the Landau expansion leads to the second-order log-periodic formula [21] 



lnp(t) « A 2 + <B 2 + C 2 cos 



■ (3) 



A third-order formula has also been given in [11] which derives from the addition of a third-order 
term in the Landau expansion. We do not write this formula explicitely here as we shall not need 
it for the present analysis. Equation (3) describes a transition from an angular log-frequency ui (for 
r < At) to a different angular log-frequency lo + A u (for At < r). Note that expression (3) reduces 
to equation (1) in the limit Af — > +oo. Using these three formulas, a prediction was published in 
January 1999 on the behavior of the Japanese stock market in the following two years [11], that 
have been remarkably successful [12]. 

The present situation of Japan does not seem any more very different from that of the US 
after the burst of the "new economy" bubble in March- April 2000 [13] (paralleling the end of the 
Japanese bubble in January 1990) and the cascade of discoveries, not yet fully unveiled in their full 
extent, of creative accounting of companies striving to look good in the eyes of analysts rather than 
to build strong fundamentals (paralleling the discovery of a surprising amount of bad loans held by 
Japanese banks). This remark takes a forceful meaning when looking at Fig. 1, which compares the 
behavior of the Japanese Nikkei index and of the US S&P500 index under a time shift of 1 1 years. 
The three fits of the Nikkei index, shown in Fig. 1 as undulating lines, use the three mathematical 
expressions discussed above. The dashed line is the simple log -periodic formula (1); the dotted 
line is the improved nonlinear log -periodic formula (3) developed in [21] and also used for the 
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October 1929 and October 1987 crashes over 8 years of data; the continuous line is the extension 
of the previous nonlinear log-periodic formula to the third-order Landau expansion developed in 
Ref. [11]. This last more sophisticated mathematical formula predicts the transition from an angular 
log-frequency oj (for r < A t ) to another angular log-frequency oj + A w for A t < r < A' t and to a 
third angular log-frequency oj + A w + A^ (for A' t < r). 

In the next section, we use the insight provided by the theory of critical herding [23, 10, 16, 9, 
22, 20] to analyze the S&P500 2000-2002 antibubble. We perform a battery of tests, starting with 
parametric fits of the index with two of the above log-periodic power law formulas, followed by the 
so-called Shank's transformation applied to characteristic times. We then present two spectral anal- 
ysis, the Lomb periodogram applied to the parametrically detrended index and the non-parametric 
(H, g)-analysis of fractal signals. These different approaches complement each other and confirm 
the remarkably strong presence of log -periodic structures. We also detect a significant second-order 
harmonic which provides a statistically significant improvement of the description of the data by 
the theory, as tested using the statistical theory of nested hypotheses. Section 3 offers an analysis of 
what could be the future evolution of the S&P500 index over the next two years by comparing the 
predictions of three formulas. The predictions are found to be robust and consistent. We conclude 
by speculating in section 4 on possible consequences as well as projections further ahead. 

2 Analysis of the S&P500 2000-2002 antibubble 

2.1 Theoretical foundations 

The analysis presented below relies on a general theory of financial crashes and of stock market 
instabilities developed in a series of works (see [23, 10, 16, 9, 22, 20] and references therein). 
The main ingredient of the theory is the existence of positive feedbacks in stock markets as well 
as in the economy. Positive feedbacks, i.e., self-reinforcement, refer to the fact that, conditioned 
on the observation that the market has recently moved up (respectively down), this makes it more 
probable to keep it moving up (respectively down), so that a large cumulative move may ensue. 
The concept of "positive feedbacks" has a long history in economics and is related to the idea of 
"increasing returns"- which says that goods become cheaper the more of them are produced (and 
the closely related idea that some products, like fax machines, become more useful the more people 
use them). "Positive feedback" is the opposite of "negative feedback", a concept well-known for 
instance in population dynamics: the larger the population of rabbits in a valley, the less they have 
grass per rabbit. If the population grows too much, they will eventually starve, slowing down their 
reproduction rate which thus reduces their population at a later time. Thus negative feedback means 
that the higher the population, the slower the growth rate, leading to a spontaneous regulation of 
the population size; negative feedbacks thus tend to regulate growth towards an equilibrium. In 
contrast, positive feedback asserts that the higher (respectively lower) the price or the price return 
in the recent past, the higher (respectively lower) will be the price growth in the future. Positive 
feedbacks, when unchecked, can produce runaways until the deviation from equilibrium is so large 
that other effects can be abruptly triggered and lead to rupture or crashes. Alternatively, it can give 
prolonged depressive bearish markets. 

There are many mechanisms leading to positive feedbacks including hedging derivatives, in- 
surance portfolios, investors' over-confidence, imitative behavior and herding between investors. 
Such positive feedbacks provide the fuel for the development of speculative bubbles as well as anti- 
bubbles [11, 12], by the mechanism of cooperativity, that is, the interactions and imitation between 
traders may lead to collective behaviors similar to crowd phenomena. Different types of collective 
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regimes are separated by so-called critical points which, in physics, are widely considered to be one 
of the most interesting properties of complex systems. A system goes critical when local influences 
propagate over long distances and the average state of the system becomes exquisitely sensitive to a 
small perturbation, i.e. different parts of the system become highly correlated. Another characteris- 
tic is that critical systems are self-similar across scales: at the critical point, an ocean of traders who 
are mostly bearish may have within it several continents of traders who are mostly bullish, each 
of which in turns surrounds seas of bearish traders with islands of bullish traders; the progression 
continues all the way down to the smallest possible scale: a single trader [25]. Intuitively speaking, 
critical self-similarity is why local imitation cascades through the scales into global coordination. 
Critical points are described in mathematical parlance as singularities associated with bifurcation 
and catastrophe theory. At critical points, scale invariance holds and its signature is the power law 
behavior of observables. 

The last ingredient of the model is to recognize that the stock market is made of actors which 
differ in size by many orders of magnitudes ranging from individuals to gigantic professional in- 
vestors, such as pension funds. Furthermore, structures at even higher levels, such as currency 
influence spheres (US$, Euro, YEN ...), exist and with the current globalization and de-regulation 
of the market one may argue that structures on the largest possible scale, i.e., the world economy, 
are beginning to form. This means that the structure of the financial markets have features which re- 
sembles that of hierarchical systems with "traders" on all levels of the market. Of course, this does 
not imply that any strict hierarchical structure of the stock market exists, but there are numerous 
examples of qualitatively hierarchical structures in society. Models of imitative interactions on hi- 
erarchical structures predict that the power law behavior can be decorated by so-called log-periodic 
corrections. Indeed, through the existence of prefered scales in a discrete hierarchy, or a discrete 
cascade of instabilities [19] or the existence of a competition between positive and negative nonlin- 
ear feedbacks [7], the scale invariance characterizing critical points may be partially broken into a 
discrete scale invariance, that is, the observable is invariant with respect to changes of scales which 
are integer powers of a fundamental scaling ratio A. It is easy to show that log-periodicity as given 
by the term C\T a cos [oj In (r) + fa] of expression (1) is the signature of discrete scale invariance: 
the term cos [u> In (r) + fa] reproduces itself each time lnr changed by 2-k/uj, that is, each time r 
is multiplied by A = exp[27r/cj]. This theory predicts robust and universal signatures of speculative 
phases of financial markets, both in accelerating bubbles as well as in decelerating anti-bubbles. 
These precursory patterns have been documented for essentially all crashes on developed as well as 
emergent stock markets, on currency markets, on company stocks, etc. 

2.2 Log-periodic fits 

We use equations (1) and (3) to fit the logarithm of the S&P500 index over an interval starting at 
time tgtart and ending at Aug. 24, 2002. The justification of the use of the logarithm of the price is 
presented in [10]. The choice of i s tart is not completely obvious. It is clear that it should be close 
to the peak of the S&P500 index in 2000 but cannot be expected to be exactly coincident with the 
time of the peak due to finite-size and other effects spoiling the validity of the log-periodic power 
law. We address this problem in two ways. First, we scan t s tart and select 10 time series, starting 
respectively at i start = 1st March 2000, 1st April 2000, 1st December 2000. The comparison of 
the fits obtained for these 10 time series will give a sense of their sensitivity with respect to t s t a rt- 
Second, we notice that we can generalize the definition of r given by (2) into 

r = \t-t c \. (4) 
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While definition (2) together with the logarithmic as well as power law singularities associated 
with formulas (1) and (3) imposes that t c < t s tart f° r an anti-bubble, the definition (4) allows for 
the critical time t c to lie anywhere within the time series. In that case, the part of the time series 
for t < t c corresponds to an accelerating "bubble" phase while the part t > t c corresponds to a 
decelerating "anti-bubble" phase. Definition (4) has thus the advantage of introducing a degree of 
flexibility in the search space for t c without much additional cost. In particular, it allows us to avoid 
a thorough scanning of t s tart since the value of t c obtained with this procedure is automatically 
adjusted without constraint. 

There are two potential problems associate with this new procedure (4). First, it assumes that 
the anti-bubble is always associated with a bubble which, in addition, has the same t c . Second, it as- 
sumes that the bubble and anti-bubble are exactly symmetric around t c , that is, the same parameters 
characterize the index evolution for t < t c and for t > t c . For the cases relevant to the present study, 
these two problems are quite minor and can be neglected because t c is always found close to t s tart 
(within the time series for t s tart prior to August 2000 and anterior to the time series otherwise). Our 
comparison with fits using (2) shows that the new procedure provides significantly better and more 
stable fits, with in particular a value of t c very weakly sensitive to t s tart- The parameters of the 
fits with the first-order formula using (2) or (4) are presented in Table 1 , using subscripts and 1 
respectively. The fits with the first-order formula (1) with the definition (2) are unstable, are quite 
sensitive to i start and have on average much larger values of their residuals % (xo > Xi)- Com- 
paring parameters with subscripts 1 and 2, one can see that applying the symmetry condition (4) 
improves the quality of fit remarkably. We stress however that this improvement (4) is not crucial 
and our results reported below remain robust with the definition (2). It would also be quite easy to 
relax the constraints of (4) by replacing r = \t — t c \ by an asymmetric function allowing in addition 
for a plateau or time lag. Since this would involve additional and poorly constrained additional 
parameters, we do not pursue this possibility here. 

The results of the fits of the logarithm of the S&P 500 index from t staT t to August, 24, 2002 
with Eqs. (1) and (3) using the improved scheme (4) are presented in figure 2 and in Table 1 under 
the subscripts 1 and 2 respectively. The ten oscillating curves correspond to the ten best fits, one for 
each of the 10 chosen values of t start from March to December 2000. Over the approximately two 
years period available for the S&P500 anti-bubble, we find that the two formulas give essentially the 
same results and the same predictions for the following year. This is reflected quantitatively by the 
facts that xi ~ X2 and that the parameters A t 's are extremely large, in which case expression (3) 
reduces to equation (1) in the limit A t — ► +oo. The top (respectively bottom) panel corresponds 
to using equation (1) (respectively (3)). The curves are shown as continuous line in their fitting 
interval and as dotted line in their extrapolation to the future. Note the very robust nature of the 
solutions obtained for the ten choices of t st&rt , which essentially all agree in their parameters and 
in their prediction of the future evolution. 

To sum up, varying the starting date t s tart of the fitted time window over a 10-month period 
and using two formulas, we confirm that a single log-periodic power law describes very well the 
S&P500 anti-bubble since around mid-2000 According to the values of t Ct i listed in Table 1, the 
critical t c is around 09-Aug-2000. This is consistent with the fact that the fit residuals obtained with 
i start = Ol-Mar-2000, Ol-Apr-2000, Ol-Oct-2000, Ol-Nov-2000 and Ol-Dec-2000 are significantly 
larger (xi > 3.3) than the residuals for the other values of i s tart- It is natural that fits with i start 
close to t c have smaller fit residuals. The fits with i start = Ol-May-2000, Ol-Jun-2000, Ol-Jul-2000, 
Ol-Aug-2000 and Ol-Sep-2000 indeed exhibut smaller residuals. This conclusion is also supported 
by the values of ui and of a which are basically constant for these five starting dates. Combining all 
the information shown in Table 1, we have the critical time t c = 09-Aug-2000 ±5 days, the angular 
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log-frequency oj = 10.30 ± 0.17 and the exponent a = 0.69 ± 0.02. The fits with the second-order 
formula (3) gives similar results. 



2.3 Analysis using the Shank's transformation on a hierarchy of characteristic times 

The fundamental idea behind the appearance of log-periodicity is the existence of a hierarchy of 
characteristic scales. Reciprocally, any log-periodic pattern implies the existence of a hierarchy of 
characteristic time scales. This hierarchy of time scales is determined by the local positive maxima 
or minima of the function such as log[p(i)]. For the S&P500 index anti-bubble, let us consider 
the times t n at which the S&P500 index reached a local minimum. As seen in figure 2, there is 
a clear sequence of sharp minima. We number them from the most recent one t\ = 23-M-2002, 
t 2 = 21-Sep-2001, t 3 = 04-Apr-2001, t 4 = 20-Dec-2000 up to the earliest one at t 5 = 12-Oct- 
2000, which is still obvious. According to the prediction of log-periodicity, the spacing between 
successive values of t n approaches zero as a geometric series as n becomes large and t n converges 
to t c . We have t\ — t 2 = 305 days, t 2 — t% = 170 days, t 3 — t^ = 105 days and t± — t$ = 69 days. 

Specifically, log-periodicity predicts that the times t n are organized in a geometric time series 
such that 

r 

tn *c = -y^ i (5) 

where r sets the time unit and 

A = e~ (6) 
is the prefered scaling ratio. The relation (5) leads to 

tn tc t n 



tn+l — t c t n+ \ — t n+ 2 



= A , (7) 



which is a signature of the discrete self-similarity of the log-periodic oscillations. Using the previ- 
ously determined dates t±, t§, we obtain 

(8) 
(9) 



h 


~t 2 


= 1.79 


*2 






*2 


-h 


= 1.62 


h 


-u 




t 3 


-u 


= 1.52 


U 


-t 5 





(10) 

These three values given by (8,9,10) are compatible but smaller than the value of A = 1.9 ± 0.1 
deduced from the log -periodic fits shown in Table 1 . Note that, according to the theory, the hierarchy 
of scales t c — t n are not universal but depend upon the specification of the system. What is expected 
to be universal are the ratios < " +1 T <c = A. 

From three successive observed values of t n , say t n , t n+ i and t n+2 , we can obtain an estimation 
of the critical time by the following formula 

+ _ t n+1 - t n+2 t n 



2tn+l — t n — t n+2 



This relation applies the so-called Shanks transformation [1] to accelerate the convergence of se- 
ries. In the case of an exact geometrical series, three terms are enough to converge exactly to the 
asymptotic value t c . Notice that this relation is invariant with respect to an arbitrary translation in 



6 



time. Applying (11) with ti, <2, t% gives t c = 02-Sep-2000. Applying (11) with t2, t%, t& gives t c = 
03-Jul-2000. Applying (11) with £3, £4, £5 gives t c = Ol-Jun-2000. These back-predictions for t c 
are compatible with the value of t c = 09-Aug-2000 ±5 days given in Table 1 determined by the 
log-periodic fits. 

In addition, the geometric structure of the time series t\, £2, ■ ■ ■ is such that the next time t n+3 
can be obtained from the first three ones by 

tn + i + t^ + 2 - tntn+2 ~ t n +\t n +2 
tn+3 = " — • Ua) 

Since time is measured backward as n increases, we are interested in the time to at which the next 
future minimum occurs. For this, we use formula (12) and put n = to get 

= £ + %Z ^ ~ ^ „ 21 Jan 2004. (13) 

t 2 -t 3 

While this Shank analysis has the advantage of simplicity and of having an obvious geomet- 
rical interpretation, its weakness lies in using a geometric set of characteristic times {£„} whose 
identification may be quite subjective. In the present case, the minima are so distinctive and sharp 
that there is no ambiguity. But,the use of only three characteristics dates makes more serious the 
sensitivity to noise and is of course bounded to lead to less precise fits and predictions that the full 
parametric fits reported in the previous section 2.2. 



2.4 Spectral analysis 

While the two previous analyses are suggestive, the parametric nature of the first one and the limited 
power of the second one requires additional tests of the reported log-periodicity. In this goal, we 
now turn to objective approaches for the detection of log-periodicity by applying a spectral Lomb 
analysis [17]. The Lomb analysis is a spectral analysis designed for unevenly sampled data which 
gives the same results as the standard Fourier spectral analysis. We apply this spectral analysis 
to two types of signals. Following [8], the first one is obtained by detrending the logarithm of 
the S&P500 index, using the power law (without log-periodicity) with the exponent a determined 
from the previous fits with formula (1). The time series of the residuals of the simple power law fit 
should be a pure cosine of In r if log-periodicity was perfect. We also use a recently developed non- 
parametric approach called the (H, g)-analysis, which has been successfully applied to financial 
crashes [29] and critical ruptures [28] for the detection of log-periodic components. 



2.4.1 Parametric detrending approach 

We first construct the following detrended quantity which is defined using Eq. (1) by: 



' Ai - \np(t)~ 




^ln 


~A 1 - In p(t)~ 


[ (t-t c ) a _ 




[ (t-t c ) a \ 



(14) 



where the bracket refers to the sample average. Table 1 shows that the values of A\ and a are 
quite stable and approximately equal to 7.33 and 0.69 for different i s tart- In order to investigate the 
impact of the different choice of t c , we first construct lnp(t) — A\ as a function of ln(t — tc) for 
different t c by fixing A\ = 7.33. We then detrend it directly by determining the exponent a from a 
linear fit, whose residuals r(t) define the time series to be analyzed for log-periodicity. 
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Figure 3 shows the residual time series r(t) as a function of ln(i — t c ) for three different critical 
times t c : 15-Jul-2000 (top panel), Ol-Aug-2000 (mid panel) and 15-Aug-2000 (bottom panel). The 
log-periodic undulations are clearly visible. We then perform the spectral Lomb analysis on these 
residuals. The three corresponding Lomb periodograms shown in Fig. 4 are very consistent with 
each other. They all exhibit an extremely strong and significant peak close to the log-frequency 
/ = uj/2it = 1.7 with an amplitude larger than 140. Its harmonic shown as the downward pointing 
arrow is significant. As we shall discuss later in section 2.5, the existence of harmonics have been 
shown to be important factors for qualifying log-periodicity [27, 30]. The presence of this harmonic 
is thus taken as a confirmation of the presence of log-periodicity. The third peak at / = 2.5 — 3 
is also significant but less well-constrainted. The inset of Fig. 4 magnifies the Lomb periodogram 
in the neighborhood of the largest peaks. The two highest Lomb peaks for t c = Ol-Aug-2000 and 
t c = 15-Aug-2000 are significantly higher than the highest peak for t c = 15-M-2000, confirming 
that the critical time is somewhere between Ol-Aug-2000 and 15-Aug-2000 (recall that section 2.2 
found t c f» 09-Aug-2000 ±5 days). As the number of the data points used for the Lomb analysis is 
the same in all analyses throughout this paper, this warrants a comparison of the Lomb peak heights 
for different t c 's to establish their significance levels [26]. The log -periodic frequency of the largest 
peak for t c = Ol-Aug-2000 is / = 1.71, corresponding to u = 2irf = 10.7, which is in reasonable 
agreement with the value a; = 10.30 ±0.17 reported in Table 1. 

To further assess the sensitivity with respect to the choice of t c , we choose 21 different t c 
evenly spaced in the time interval from 15-Jul-2000 to 13-Sep-2000. For each t c , we perform the 
same analysis as above and obtain the highest Lomb peak and its associated log-frequency. The 
results are shown in Fig. 5. The log-frequency is found to decrease with t c . This is due to the 
fact that, moving the critical time forward (respectively backward), the other end of the time series 
will decelerate (respectively accelerate) the log-periodic oscillations. Fig. 5 confirms that the best 
critical time t c falls somewhere in the first half of August 2000. The corresponding log-frequencies 
are / = 1.63 - 1.72 (lo = 10.2 - 10.7), in agreement with Table 1. 

It is interesting that the most relevant angular log-frequency uo k, 10 fitted on the S&P500 
index is found very close to twice the value k, 5 found previously for the Nikkei index. Actually, a 
small but noticeable peak at this value / « 5/27T « 0.80 can be seen in Fig. 4, strengthening the 
analogy quantitatively. It may thus be surmised that both the Nikkei and the S&P500 indices are 
characterized by a universal discrete hierarchy of (angular) log-periodic frequencies, all harmonics 
of a fundamental angular log-frequency close to 5. Other systems have previously exhibited the 
curious fact, also observed when comparing the S&P500 to the Nikkei indices, that the higher- 
order harmonics may have an amplitude larger than the fundamental value [15, 27, 30], as observed 
for the S&P500 index. 



2.4.2 Non-parametric (H, g)-analysis 

The (H, q) -analysis [28, 29] is a generalization of the (/-analysis [2, 3], which is a natural tool for 
the description of discretely scale invariant fractals. The (H, g)-derivative is defined as 

D H f(x) A f(x)-f(qx) 

[(l-q)x] H ' UJ 

The special case H = 1 recovers the normal ^-derivative, which itself reduces to the normal deriva- 
tive in the limit q — > 1 _ . There is no loss of generality by constraining q in the open interval (0, 1) 
[28]. We apply the (H, g)-analysis to verify non-parametrically the existence of log-periodicity by 
taking f(x) = lnp(t) and x = t — t c . The advantage of the (H, g)-analysis is that there is no need 
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of detrending as done in the previous section 2.4.1. Such detrending is automatically accounted for 
by the finite difference and the normalization by the denominor. 

Fig. 6 shows the (H, q) -derivatives of the logarithm of S&P500 index as a function f \n(t — t c ) 
for three choice of t c : 15-M-2000 with H = and q = 0.8 in the top panel, 02-Aug-2000 with 
H = 0.5 and q = 0.7 in the mid panel, and 16-Aug-2000 with H = 0.4 and q = 0.7 in the bottom 
panel. Each of these pairs of (H, q) is the optimal pair [28, 29] corresponding to the most significant 
peak among all Lomb periodograms for each t c . The log-periodic undulations are clearly visible 
to the naked eyes. The Lomb periodograms of the (H, q) -derivatives shown in Fig. 6 are presented 
in Fig. 7. The three highest Lomb peaks are even more significant than those in Fig. 4. The inset 
shows the magnified Lomb peaks. The highest Lomb peak is obtained for t c = 02-Aug-2000, which 
corresponds to the log-frequency / « 1.71. 

To test the robustness of the (H, q) -analysis for the detection of log-periodicity, we scan H 
from —1 to 1 with spacing 0.1 and q from 0.1 to 0.9 with spacing 0.1 for each critical t c . Figure 8 
shows the log-frequency / as a function of H and q for t c = 16-Aug-2000. The existence of a flat 
plateau at / = 1.62 ± 0.07 for most of the pairs (H, q) confirm the existence of log-periodicity. The 
five smaller log-frequencies below the plateau correspond to the spurious values stemming from 
the most probable effect of noise on power laws [6] and should be discarded. The three higher log- 
frequencies above the plateau probably stem from the interaction between high-frequency noise and 
the second harmonics. 

We then apply the non-parametric (H, q) -analysis to the S&P500 index for 21 different choices 
of the critical t c . For each given t c , we take the average of the Lomb periodograms for all 21 x 9 
pairs of (H,q). The amplitude of the highest peak in each averaged Lomb periodogram is plotted 
as a function of t c in the lower panel of Fig. 9. Their associated log-frequency shown in the upper 
panel of Fig. 9 with error bars is estimated from the average height of the plateaus such as the one 
seen in Fig. 8. This log-frequency slightly decreases with t c . There are two clear humps in the 
lower panel around late July and mid August 2000. The hump around mid August is higher than the 
other one. The corresponding log-frequencies / 1.60 — 1.70 are compatible with those reported 
in Table 1. 

2.5 Role of log-periodic harmonics 

The spectral Lomb analyses reported above (see figures 4 and 7) as well as a the visual structure 
of the S&P500 time series suggest the presence of a rather strong harmonic at the angular log- 
frequency 2u. The possible importance of harmonics in order to qualify log-periodicity is made 
also more credible by recent analyses of log -periodicity in hydrodynamic turbulence data [27, 30] 
which have demonstrated the important role of higher harmonics in the detection of log-periodicity. 

We thus revisit the parametric log-periodic fits of section 2.2 with formula (1) using (4) to 
include the effect of an harmonic at the angular log-frequency 2uj. In this goal, we postulate the 
formula 

hip (t) » A + Br a + Cr° cos [to In (r) + fa] + Dr a cos [2u In (r) + fa] , (16) 

which differs from equation (1) by the addition of the last term proportional to the amplitude D. 
This formula has two additional parameters compared with (1), the amplitude D of the harmonic 
and its phase fo. We follow the fit procedure of Ref. [31] with minor modifications, which itself 
adapts the slaving method of [8,9]. By rewriting Eq. (16) aslnp(t) = A+Bf(t) + Cg(t) + Dh(t), 
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we obtain a system of 4 linear equations for the four variables A, B, C and D: 
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(17) 



where = f\ = f(U), gi = g(U) and hi = h(ti). Solving analytically this system allows 

us to slave the four parameters A, B, C and D to the other parameters in the search for the best fit. 
With this approach, we find that the search of the optimal parameters is very stable and provides fits 
of very good quality in spite of the remaining five free parameters. 

The results are listed in Table 2 and depicted in figure 10. The fit residuals are reduced consid- 
erably compared with the fits reported in Table 1 . The improvement of the fits are obvious when 
comparing Fig. 10 with figure 2. Note also that \D\ < \C\ (\C\ « 5|D|) which is in agreement with 
the fact that the spectral peak of the fundamental log-frequency is much higher than the peak of the 
second harmonic approximately by a factor 5, as shown in Fig. 4 and Fig. 7. 

We have also tested whether the addition of a third log-frequency around / « 2.8 (which is 
not a third harmonic), as suggested from the spectral Lomb analyses shown in figures 4 and 7), 
could improve and/or modify the fit. We found a slight but non-significant reduction of the root- 
mean-square error with negligible modification of the fit, suggesting that this log-frequency is due 
to noise. 

Since expression (16) contains the formula (1) as a special case D = 0, we can use Wilk's 
theorem [18] and the statistical methodology of nested hypotheses to assess whether the hypothesis 
that D = can be rejected. Therefore, the null hypothesis and its alternative are 

1. flo: D = 0; 

2. Hn D^O; 

The method proceeds as follows. By assuming a Gaussian distribution of observation errors (resid- 
uals) at each data point, the maximum likelihood estimation of the parameters amounts exactly to 
the minimization of the sum of the square over all data points (of number n) of the differences Sj (i) 
between the mathematical formula and the data [17]. The standard deviation <jj with j = 0, 1 of 
the fits to the data associated respectively with (1) and (16) is given by 1/n times the sum of the 
squares over all data points of the differences 5^ (i) between the mathematical formula and the 
data, estimated for the optimal parameters of the fit. The log-likelihoods corresponding to the two 
hypotheses are thus given by 

Lj = — n In \p2/K — n lno"j — n/2 , (18) 
where the third term results from the product of Gaussians in the likelihood, which is of the form 

oc nexp[-(4 o) (i)) 2 /2^ 2 ] = exp[-n/2] , 



i=i 



from the definition aj = (1/n) E^it^W] 2 - Then, according to Wilk theorem of nested hy- 
potheses, the log-likelihood-ratio 



T = -2(Lq - = 2n(lncji - ma ) , 



(19) 
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is a chi-square variable with k degrees of freedom, where k is the number of restricted parameters 
[5]. In the present case, we have k = 1. 

The Wilk test thus amounts to calculate the probability that the obtained value of T can be 
overpassed by chance alone. If this probability is small, this means that chance is not a convincing 
explanation for the large value of T which becomes meaningful. This implies a rejection of the 
hypothesis that D = is sufficient to explain the data and favor the fit with D / as statistically 
significant. 

In our test, the beginning of the fitted data set is fixed at 09-Aug-2000, while the end of the data 
set varies from Ol-Jan-2001 to 24-Aug-2002. The results of the Wilk test are presented in Table 3. 
Increasing the number of points decreases the probability that the obtained probability to overpass T 
may result from chance, and thus increases the statistical significance of the fit with Eq. (16). Since 
the assumption of Gaussian noise is most probably an under-estimation of the real distribution of 
noise amplitudes, the very significant improvement in the quality of the fit brought by the use of 
Eq. (16) quantified in Table 3 provides most probably a lower bound for the statistical significance 
of the hypothesis that D should be chosen non-zero, above the 99.8% confidence level. Indeed, 
a non-Gaussian noise with a fat-tailed distribution would be expected to decrease the relevance of 
competing formulas, whose performance could be scrambled and be made fuzzy. The clear and 
strong result of the Wilk test with assumed Gaussion noises thus confirm a very strong significance 
of Eq. (16). 

Strengthened by this analysis of the strong relevance of the second harmonics at 2lj, we revisit 
the Nikkei index and fit it in the first 2.6 years of its decay starting in January 1990 using (16) 
to test whether the second harmonics is also important for the Nikkei index. The fit is compared 
with the log -price in Fig. 11. We find indeed an impressive improvement, as the mean square 
error x = 0.0457 is significantly smaller than the mean square error x = 0.0535 obtained with 
expression (1) and whose fit is shown in Fig. 1. 

3 Discussion and prediction 

Starting from a visual analogy with the Nikkei index shifted by 11 years, the first point of the anal- 
yses presented above is to have established with strong significance the existence of an anti-bubble 
followed by the S&P500 index approximately since July- August 2000. This anti -bubble is charac- 
terized by an overall power law decay of the index decorated by strong log-periodic oscillations. 

Following the analogy with the trajectory of the Nikkei index 11 years earlier, the second point 
is that a comparison between the fits obtained with equations (1) and (3) shows that the S&P500 
index has not yet entered into the second phase in which the angular log-frequency may start its 
shift to another value, as did the Nikkei index after about 2.5 years of its decay. We may expect 
this to occur in the future. Not being able to estimate directly the parameter A t controlling this 
transition due to the smallness of the S&P500 index anti-bubble duration, we can however offer the 
following guess, based on the hypothesis that the values A t and A^ given by the fit of expression 
(3) to the logarithm of the Nikkei index are reasonable estimates of those for the S&P500 index. 
We use also the parameters in the column of t star t = Aug-0 1-2000 of Table 1 for the first order 
regime and extrapolate the fitted curve to 2006 (continuous line). Using the values of the fits with 
(1) and plugging in the values of A t and A w from the Nikkei index in expression (3) gives the 
dashed line shown in Fig. 12. The crossover from the first order regime to the second order regime 
is here suggested to occur in the first half of 2004. Fig. 12 also compares the first-order fit and the 
second-order guess with formula (16) taking into account the second harmonics. Fig. 12 suggests 
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that the next broad minimum of the S&P500 index will occur in the first semester of 2004. This is 
consistent with the prediction (13) using Shank's transformation. 

The third important point is the improvement in the quality of the fits and therefore in the 
potential for predictions when adding the harmonics at 2u>, as shown in figure 10 compared with 
figure 2. Figure 2 suggests a local maximum of the S&P500 index around the end of the first quarter 
of 2003, while figure 10 refines this prediction by seeing an earlier peak before but close to the end 
of 2002. These two predictions are not in contradiction: the prediction of figure 10 shows that the 
oscillatory structure of S&P500 index implies several ups and downs in the coming year, with a 
tendency to appreciate for a while before going down again by the end of 2003. 

Ideally, we would like to combine the effect of the second-order formula (3) with the impact of 
the second harmonics described by expression (16). We do this by adding to (16) a term with the 
same structure as the one proportional to C2 in (3) with u> replaced by 2w, again fixing A t and A w 
at the values determined from the Nikkei index. Figure 13 compares the result of this fit with that 
with (16) and their extrapolation up to close to the end of 2006. These two curves provide a sense 
of the future directions of the S&P500 index and their probable degree of variability. 

4 Concluding remarks 

The growing awareness in 2002 of the crisis in the American financial system is reminiscent of the 
starting point of Japan's massive financial bubble burst more than 10 years before and of the inter- 
twining of the bad debts and bad performance of banks whose capital is invested in the shares of 
other banks, thus creating the potential for a catastrophic cascade of bankrupts. Japan has rediscov- 
ered before the US the faults of the 19th century financial system in the US in which stock markets 
were so much intertwined with their overall banking financial system, that busts and bursts oc- 
curred more than once every decade, with firms losing their credit lines and workers and consumers 
their savings and often their employment. It is often said that the 1930s depression was the last of 
the stock market and bank-induced economic collapses. The growing fuzziness between financial 
banking systems and stock markets, in part due to the innovations in information technology, has 
re-created the climate for stronger bubbles and more pronounced losses of confidence leading to 
long-lived bearish regimes possibly nucleating depressions. 

A big problem is that, in the collapse following them, policy interventions such as lowering 
interest rates, reducing taxes, government spending packages and any measure to restore investors' 
confidence may be much less effective, as discovered with the Japanese so-called liquididy trap, a 
process in which government and the central bank policy becomes essentially useless. In addition, 
loss of confidence by investors (for instance following the frauds in accounting in the US) may lead 
to a non-negligible cost to the overall economy [4], providing a positive feedback reinforcing the 
bearish climate. 

We have proposed that the trajectories of the US and Japanese stock markets could be un- 
derstood in large part by taking into account imitative and herding mechanisms, both stemming 
possibly from rational or irrational behaviors. A key ingredient entering probably in the imitative 
and herding processes is the phenomenon of investor confidence. It has recently been argued [24] 
that investor confidence can be understood far better if one assumes not that investors have rational 
expectations, but that they have what economists call "adaptive expectations." Individuals with ra- 
tional expectations predict others' behavior by focusing on their external incentives and constraints. 
In contrast, individuals with adaptive expectations predict others' behavior (including possibly the 
behavior of such an abstract "other" as the stock market) by extrapolating from the past. In addi- 
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tion, confidence and trust in the market have been shown to be subjected to history effects [24]. We 
believe that these behavioral traits provide fundamental roots underlying the validity of our analysis 
which, ultimately, can be viewed similarly as nothing but a (rather complex nonlinear) extrapolation 
of the past. 

Our theory does not describe the common "stationary" evolution of the stock market, but rather 
is specifically tailored for identifying "monsters" or anomalies (bubbles and their end) and for 
classifying their agonies. We claim that these agonies (the anti-bubbles) are mostly shaped by 
collective effects between economic and stock market agents, with their imitation and confidence 
(and lack thereof) idiosynchracies. Ultimately, our description must leave place to a recovery of the 
fundamental pricing principles (and of possible emergences of new speculative bubbles) but, before 
this, it describes the way by which collective effects control in large part the processes towards this 
recovery. 

The present study complements and makes more precise a previous one focusing at longer times 
scales, based on three pieces of evidence, namely the growth over long time scales of population, 
gross national product and stock market indices [14, 20], which issued a prediction that starting 
around 1999, a 5 to 10 years consolidation of international stock markets will occur, allowing a 
purge after the over-aggressive appetite of the preceding decade. For more than the last two years, 
this prediction has been born out. The present study confirms this impression that the US stock 
market is not yet on the verge of recovery. 

With its extraordinary and unparalleled growth, its ensuing decade-long absence of growth, its 
crowded land, its aging population, is Japan a precursor of the new regime that mankind has to shift 
to, as discussed in [14, 20]? It seems that the present qualification of the US market to be in an anti- 
bubble phase is entirely in line with these predictions. From a larger perspective and at the horizon 
of the end of the first half of this century, the behavior of these stock markets raise the following 
question: shall we learn the lessons of previous bubbles and crashes/depressions and shall we be 
able to transit to a qualitatively different organization of economic and cultural exchanges before 
the fundamental limitations of a finite earth and limited human intelligence set in? 
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Table 1: Fitted parameters using equations (1) and (3) on the S&P500 index. The parameters with 
subscript correspond to fits using equation (1) together with (2). The subscripts 1 and 2 refer to 
fits with expressions (1) and (3) respectively together with (4). The rows of xo, Xi an d X2 present 
the standard deviations (r.m.s.) of the residuals for different fits. Note that the fits with the first- 
order formula (1) with definition (2) are unstable and sensitive to £ B tart- Comparing parameters with 
subscripts 1 and 2, one can see that applying (4) improves significantly the quality of the fits. With 
this new symmetric definition (4), we find that the first-order fits using (1) are quite robust and close 
to the fits with the second-order formula (3). The bold values are discussed in the text. 
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Table 2: Parameters of the fit with equation (16) with (4) of the S&P500 index for different t sta rt- 
The fit residuals are strongly reduced compared with the fits shown in Table 1. The bold columns 
correspond to the values of t stai t giving basically the same values for t c , a and u). 
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Table 3: Likelihood-ratio test of the hypothesis that D / in Eq. (16). The beginning of the data 
set for fit is set to be fixed at 09-Aug-2000. The end of the data set varies from Ol-Jan-2001 to 
24-Aug-2002. The n column gives the number of the data set for each fit. The confidence level 
quantified by Proba decreases on average with n. There is no doubt that the Hi hypothesis of 
D^O can not be rejected for all cases in the 99.8% confidence level. 
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Figure 1: Comparison between the evolutions of the US S&P500 index from 1996 till August, 24, 
2002 (bottom and right axes) and the Japanese Nikkei index from 1985 to 1993 (top and left axes). 
The years are written on the horizontal axis (and marked by a tick on the axis) where January 1 of 
that year occurs. The dashed line is the simple log -periodic formula (1) fitted to the Nikkei index. 
The data used in this fit goes from Ol-Jan-1990 to Ol-Jul-1992 [1 1]. The parameter values are t c = 
28-Dec-1989, a = 0.38, u = 5.0, 4> = 2.59, A = 10.76, B = -0.067 and C = -0.011. The 
fit error is x = 0.0535. The dash-dotted line is the improved nonlinear log-periodic formula (3) 
developed in [21] fitted to the Nikkei index. The Nikkei index data used in this fit goes from Ol- 
Jan-1990 to 01 -Jul- 1995 [11]. The parameter values are t c = 27-Dec-1989, a = 0.38, u = 4.8, 
4> = 6.27, A t = 6954, A w = 6.5, A = 10.77, B = -0.070, C = 0.012. The fit error is X = 
0.0603. The continuous line is the fit of the Nikkei index with the third-order formula developed in 
Ref. [11]. The Nikkei index data used in the fit goes from Ol-Jan-1990 to 31-Dec-2000. The fit is 
performed by fixing t c , a and uj at the values obtained from the second-order fit and adjusting only 
A t , A' t , Ac, A' w and 0. The parameter values are A t = 1696, A' t = 5146, A u = -1.7, A^ = 40, 
<f> = 6.27, A = 10.86, B = -0.090, C = -0.0095. The fit error is X = 0.0867. In the three fits, A, 
B and C are slaved to the other variables by multiplier approach in each iteration of optimization 
search. The inset shows the 13-year Nikkei anti-bubble with the fit with the third-order formula 
shown as the continuous line. The parameter values are A t = 52414, A£ = 17425, A w = 23.7, 
A^ = 127.5, (f> = 5.57, A = 10.57, B = -0.045, C = 0.0087. The fit error is X = 0.1101. In 
all our fits, times are expressed in units of days, in contrast with the yearly unit used in [1 1]). Thus, 
the parameters B and C are different since they are unit-dependent, while all the other parameters 
are independent of the units. 
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Figure 2: The S&P500 index anti-bubble fitted from i s tart to August, 24, 2002, with the improved 
scheme (4) inserted in the two formulas (1) (upper panel) and (3) (lower panel) for different choices 
of t s tart> spanning from Mar-0 1-2000 to Dec-0 1-2000. The dotted lines show the predicted future 
trajectories. One see that the fits are robust with respect to different starting date. 



20 



1 




ln(/-f ) 

c 

Figure 3: The residuals r(t) defined in Eq. (14) as a function of ln(t — t c ). The three plots from top 
to bottom correspond respectively to t c = 15-M-2000, t c = Ol-Aug-2000 and t c = 15-Aug-2000. 
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Figure 4: Lomb periodograms of the residuals r(t) shown in Fig. 3. The highest Lomb peaks are 
very significant. The second harmonics is also visible and is indicated by a downward pointing ar- 
row. The inset shows a magnification of the Lomb periodogram in the neighborhood of the stronger 
peaks. 
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Figure 5: Amplitude of the highest Lomb peaks (lower panel) and their associated log-frequencies 
(upper panel) obtained by apply the parametric detrending approach of section 2.4.1 for 21 different 
critical t c evenly spaced in the time interval from 15-Jul-2000 to 13-Sep-2000. 
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ln(t-t c ) 

Figure 6: The (H, q) -derivatives of the logarithm of the S&P500 index as a function of ln(i — t c ) 
for three value of t c : 15-M-2000 with H = and q = 0.8 in the top panel, 02-Aug-2000 with 
H = 0.5 and q = 0.7 in the mid panel, and 16-Aug-2000 with H = 0.4 and q = 0.7 in the bottom 
panel. 
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Figure 7: Lomb periodograms of the (H, q) -derivatives shown in Fig. 6 for three different value of 
t c . The highest Lomb peaks are even more significant than those in Fig. 4. The inset shows the 
magnified Lomb peaks in the neighborhood of the maxima. 
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Figure 8: Log-frequency / as a function of H and q for t c = 16-Aug-2000. The plateau at / = 
1.62±0.07 is a signature of the robustness of the detection of log -periodicity by the (H, g)-analysis. 
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Figure 9: The highest Lomb peaks (lower panel) and their associated log-frequencies (upper panel) 
obtained by apply the non-parametric (H, (^-analysis for 21 different critical t c evenly spaced 
shown as the abscissa. The Lomb peak height and the log-frequency are obtained by averaging 
all Lomb periodograms over all pairs (H, q) defined by scanning H from —1 to 1 with spacing 0.1 
and q from 0.1 to 0.9 with spacing 0.1. 
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Figure 10: All the fitted functions using Eq. (16). The dotted lines show the predicted future 
trajectories. One sees that the fits are quite robust with respect to different starting date t staT t from 
Mar-01-2000 to Dec-01-2000. 
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Figure 11: Fit of the 2.6 first years of the Nikkei index anti-bubble from Jan-01-1990 to Jul-31- 
1992 by Eq. (16) to test for the importance of a second harmonics. The parameter values are t c = 
27-Dec-1989, a = 0.42, uj = 5.1, <pi = 5.60, 2 = 1-80, A = 10.72, B = -0.051, C = 0.0096 
and D = 0.0034. The r.m.s. error of this fit is % = 0.0457. 
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Figure 12: Prediction of a change of regime from the first order formula (1) to the second order 
formula (3). For the first order formula (1), we use the parameters corresponding to t c = Aug-01- 
2000 given in Table 1 . The second order fit (3) uses the same parameter values as the first-order 
formula with the addition that A t and A w are fixed to the values determined the corresponding 
fit to the Nikkei index extending over 5 years of data. The crossover between the two formulas 
is predicted to occur in the first semester of 2004. We also show for comparison the fit using 
expression (16) taking into account the second harmonics at the angular log-frequency 2lo, with the 
parameters given in Table 2. 
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Figure 13: Combination of the effect of the second-order formula (3) with the impact of the second 
harmonics described by expression (16) (see text for explanations). The corresponding fit (thin 
continuous line) is compared with expression (16) (second harmonic effect only in thick continuous 
line) and extrapolates them up to close to the end of 2006. 
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